{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f6d5ee62",
   "metadata": {},
   "source": [
    "# Day 4：描述统计与探索性分析（看懂数据）\n",
    "\n",
    "> 今日目标：使用描述统计、排序、相关性与简单逻辑过滤初步理解数据结构；形成 3~5 条可解释发现。\n",
    "\n",
    "学习路径：\n",
    "1. 读取与准备 (含日期解析)\n",
    "2. 描述统计 describe / 单列指标\n",
    "3. 排序与选择 top / bottom\n",
    "4. 频数与唯一值 count / unique / value_counts\n",
    "5. 相关系数 corr 与简单解释\n",
    "6. 组合条件过滤 (逻辑与或)\n",
    "7. Mini Case：提出并回答 3 个问题\n",
    "8. 发现撰写模板\n",
    "9. 小结\n",
    "10. 课后作业提示"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a02c4135",
   "metadata": {},
   "source": [
    "## 1. 读取与准备\n",
    "数据：`air_quality_timeseries.csv`，包含 5 天 × 多城市。\n",
    "\n",
    "步骤：\n",
    "1. read_csv + parse_dates\n",
    "2. 基本查看 shape / head / info\n",
    "3. 确认数值列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b9b4dc09",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "index",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "date",
         "rawType": "datetime64[ns]",
         "type": "datetime"
        },
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "province",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "PM10",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "NO2",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "SO2",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "2b4eec2c-4cd6-4120-86ae-a5060f41e7e1",
       "rows": [
        [
         "0",
         "2025-09-01 00:00:00",
         "广州",
         "广东",
         "42",
         "55",
         "19",
         "7"
        ],
        [
         "1",
         "2025-09-02 00:00:00",
         "广州",
         "广东",
         "41",
         "54",
         "18",
         "7"
        ],
        [
         "2",
         "2025-09-03 00:00:00",
         "广州",
         "广东",
         "39",
         "50",
         "17",
         "6"
        ],
        [
         "3",
         "2025-09-04 00:00:00",
         "广州",
         "广东",
         "45",
         "57",
         "20",
         "7"
        ],
        [
         "4",
         "2025-09-05 00:00:00",
         "广州",
         "广东",
         "44",
         "56",
         "19",
         "7"
        ]
       ],
       "shape": {
        "columns": 7,
        "rows": 5
       }
      },
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>city</th>\n",
       "      <th>province</th>\n",
       "      <th>PM25</th>\n",
       "      <th>PM10</th>\n",
       "      <th>NO2</th>\n",
       "      <th>SO2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2025-09-01</td>\n",
       "      <td>广州</td>\n",
       "      <td>广东</td>\n",
       "      <td>42</td>\n",
       "      <td>55</td>\n",
       "      <td>19</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2025-09-02</td>\n",
       "      <td>广州</td>\n",
       "      <td>广东</td>\n",
       "      <td>41</td>\n",
       "      <td>54</td>\n",
       "      <td>18</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2025-09-03</td>\n",
       "      <td>广州</td>\n",
       "      <td>广东</td>\n",
       "      <td>39</td>\n",
       "      <td>50</td>\n",
       "      <td>17</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2025-09-04</td>\n",
       "      <td>广州</td>\n",
       "      <td>广东</td>\n",
       "      <td>45</td>\n",
       "      <td>57</td>\n",
       "      <td>20</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2025-09-05</td>\n",
       "      <td>广州</td>\n",
       "      <td>广东</td>\n",
       "      <td>44</td>\n",
       "      <td>56</td>\n",
       "      <td>19</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        date city province  PM25  PM10  NO2  SO2\n",
       "0 2025-09-01   广州       广东    42    55   19    7\n",
       "1 2025-09-02   广州       广东    41    54   18    7\n",
       "2 2025-09-03   广州       广东    39    50   17    6\n",
       "3 2025-09-04   广州       广东    45    57   20    7\n",
       "4 2025-09-05   广州       广东    44    56   19    7"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "df = pd.read_csv('../data/air_quality_timeseries.csv', parse_dates=['date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "1b82c8cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "形状: (30, 7)\n",
      "info():\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 30 entries, 0 to 29\n",
      "Data columns (total 7 columns):\n",
      " #   Column    Non-Null Count  Dtype         \n",
      "---  ------    --------------  -----         \n",
      " 0   date      30 non-null     datetime64[ns]\n",
      " 1   city      30 non-null     object        \n",
      " 2   province  30 non-null     object        \n",
      " 3   PM25      30 non-null     int64         \n",
      " 4   PM10      30 non-null     int64         \n",
      " 5   NO2       30 non-null     int64         \n",
      " 6   SO2       30 non-null     int64         \n",
      "dtypes: datetime64[ns](1), int64(4), object(2)\n",
      "memory usage: 1.8+ KB\n",
      "None\n",
      "数值列初步均值:\n",
      "PM25    50.500000\n",
      "PM10    66.200000\n",
      "NO2     27.366667\n",
      "SO2      9.833333\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print('形状:', df.shape)\n",
    "print('info():')\n",
    "print(df.info())\n",
    "print('数值列初步均值:')\n",
    "print(df[['PM25','PM10','NO2','SO2']].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49090bfa",
   "metadata": {},
   "source": [
    "### 小练习 1\n",
    "1. 使用 head(3) 与 tail(3) 观察数据两端。\n",
    "2. 检查有无缺失 (isnull().sum())。\n",
    "(在下方自行添加单元格)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd5ae225",
   "metadata": {},
   "source": [
    "## 2. 描述统计 describe 与单列指标\n",
    "describe()：一次性给出 count / mean / std / min / 25% / 50% / 75% / max。\n",
    "也可单独：mean() / std() / median() / max() 等。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "27787427",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "index",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "float64",
         "type": "float"
        },
        {
         "name": "PM10",
         "rawType": "float64",
         "type": "float"
        },
        {
         "name": "NO2",
         "rawType": "float64",
         "type": "float"
        },
        {
         "name": "SO2",
         "rawType": "float64",
         "type": "float"
        }
       ],
       "ref": "f66a9a0c-3200-4b66-9765-c2f15549552e",
       "rows": [
        [
         "count",
         "30.0",
         "30.0",
         "30.0",
         "30.0"
        ],
        [
         "mean",
         "50.5",
         "66.2",
         "27.366666666666667",
         "9.833333333333334"
        ],
        [
         "std",
         "9.18675958780559",
         "11.850796568600138",
         "6.327190002942328",
         "2.408080267497876"
        ],
        [
         "min",
         "37.0",
         "49.0",
         "17.0",
         "6.0"
        ],
        [
         "25%",
         "42.25",
         "55.25",
         "20.25",
         "7.0"
        ],
        [
         "50%",
         "50.5",
         "67.0",
         "29.0",
         "10.5"
        ],
        [
         "75%",
         "58.0",
         "75.75",
         "31.0",
         "12.0"
        ],
        [
         "max",
         "66.0",
         "86.0",
         "38.0",
         "13.0"
        ]
       ],
       "shape": {
        "columns": 4,
        "rows": 8
       }
      },
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PM25</th>\n",
       "      <th>PM10</th>\n",
       "      <th>NO2</th>\n",
       "      <th>SO2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>30.00000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>30.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>50.50000</td>\n",
       "      <td>66.200000</td>\n",
       "      <td>27.366667</td>\n",
       "      <td>9.833333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>9.18676</td>\n",
       "      <td>11.850797</td>\n",
       "      <td>6.327190</td>\n",
       "      <td>2.408080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>37.00000</td>\n",
       "      <td>49.000000</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>42.25000</td>\n",
       "      <td>55.250000</td>\n",
       "      <td>20.250000</td>\n",
       "      <td>7.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>50.50000</td>\n",
       "      <td>67.000000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>10.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>58.00000</td>\n",
       "      <td>75.750000</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>12.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>66.00000</td>\n",
       "      <td>86.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>13.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           PM25       PM10        NO2        SO2\n",
       "count  30.00000  30.000000  30.000000  30.000000\n",
       "mean   50.50000  66.200000  27.366667   9.833333\n",
       "std     9.18676  11.850797   6.327190   2.408080\n",
       "min    37.00000  49.000000  17.000000   6.000000\n",
       "25%    42.25000  55.250000  20.250000   7.000000\n",
       "50%    50.50000  67.000000  29.000000  10.500000\n",
       "75%    58.00000  75.750000  31.000000  12.000000\n",
       "max    66.00000  86.000000  38.000000  13.000000"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "desc = df[['PM25','PM10','NO2','SO2']].describe()\n",
    "desc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9247693a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PM2.5 平均=50.50 标准差=9.19 范围=(37,66)\n"
     ]
    }
   ],
   "source": [
    "# 单列详细例子：PM25\n",
    "pm25_mean = df['PM25'].mean()\n",
    "pm25_std = df['PM25'].std()\n",
    "pm25_min = df['PM25'].min()\n",
    "pm25_max = df['PM25'].max()\n",
    "print(f'PM2.5 平均={pm25_mean:.2f} 标准差={pm25_std:.2f} 范围=({pm25_min},{pm25_max})')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "677ac821",
   "metadata": {},
   "source": [
    "解读提示：标准差大 → 波动大；极差 (max - min) → 离散程度；中位数 vs 均值 可看是否偏斜。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "978a0b79",
   "metadata": {},
   "source": [
    "### 小练习 2\n",
    "1. 计算 PM10 的中位数 median 与极差。\n",
    "2. 估算 NO2 与 SO2 哪个波动更大 (比较标准差)。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53f3af97",
   "metadata": {},
   "source": [
    "## 3. 排序与选择 top/bottom\n",
    "按某列排序常见：sort_values(by='列', ascending=False)。\n",
    "案例：找出 PM25 最高的 5 条记录 (city+date+PM25)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0a469993",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "index",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "date",
         "rawType": "datetime64[ns]",
         "type": "datetime"
        },
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "81d21fdd-2706-431d-9863-f9275c60671d",
       "rows": [
        [
         "9",
         "2025-09-05 00:00:00",
         "北京",
         "66"
        ],
        [
         "6",
         "2025-09-02 00:00:00",
         "北京",
         "65"
        ],
        [
         "8",
         "2025-09-04 00:00:00",
         "北京",
         "62"
        ],
        [
         "18",
         "2025-09-04 00:00:00",
         "成都",
         "62"
        ],
        [
         "5",
         "2025-09-01 00:00:00",
         "北京",
         "60"
        ]
       ],
       "shape": {
        "columns": 3,
        "rows": 5
       }
      },
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>city</th>\n",
       "      <th>PM25</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2025-09-05</td>\n",
       "      <td>北京</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2025-09-02</td>\n",
       "      <td>北京</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2025-09-04</td>\n",
       "      <td>北京</td>\n",
       "      <td>62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>2025-09-04</td>\n",
       "      <td>成都</td>\n",
       "      <td>62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2025-09-01</td>\n",
       "      <td>北京</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date city  PM25\n",
       "9  2025-09-05   北京    66\n",
       "6  2025-09-02   北京    65\n",
       "8  2025-09-04   北京    62\n",
       "18 2025-09-04   成都    62\n",
       "5  2025-09-01   北京    60"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top5_pm25 = df.sort_values(by='PM25', ascending=False).head(5)[['date','city','PM25']]\n",
    "top5_pm25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "91ab82ef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "float64",
         "type": "float"
        }
       ],
       "ref": "d84b9900-4d17-408f-8616-d96570c73b1b",
       "rows": [
        [
         "北京",
         "62.2"
        ],
        [
         "成都",
         "59.2"
        ],
        [
         "重庆",
         "55.4"
        ],
        [
         "上海",
         "45.0"
        ],
        [
         "广州",
         "42.2"
        ],
        [
         "杭州",
         "39.0"
        ]
       ],
       "shape": {
        "columns": 1,
        "rows": 6
       }
      },
      "text/plain": [
       "city\n",
       "北京    62.2\n",
       "成都    59.2\n",
       "重庆    55.4\n",
       "上海    45.0\n",
       "广州    42.2\n",
       "杭州    39.0\n",
       "Name: PM25, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 也可以按城市平均 PM25 排序：\n",
    "city_pm25_mean = df.groupby('city')['PM25'].mean().sort_values(ascending=False)\n",
    "city_pm25_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "391c1f60",
   "metadata": {},
   "source": [
    "### 小练习 3\n",
    "1. 找出 PM10 平均值最高的前 3 个城市。\n",
    "2. 反向：PM25 平均值最低的 2 个城市。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3521a71e",
   "metadata": {},
   "source": [
    "## 4. 频数与唯一值\n",
    "工具：nunique() / unique() / value_counts()。\n",
    "示例：统计出现的省份数量与各省份记录数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1bc7aa99",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "省份数量: 6\n"
     ]
    },
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "province",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "count",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "c6e34fab-af50-4936-b577-6061061a6a79",
       "rows": [
        [
         "广东",
         "5"
        ],
        [
         "北京",
         "5"
        ],
        [
         "上海",
         "5"
        ],
        [
         "四川",
         "5"
        ],
        [
         "浙江",
         "5"
        ],
        [
         "重庆",
         "5"
        ]
       ],
       "shape": {
        "columns": 1,
        "rows": 6
       }
      },
      "text/plain": [
       "province\n",
       "广东    5\n",
       "北京    5\n",
       "上海    5\n",
       "四川    5\n",
       "浙江    5\n",
       "重庆    5\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "province_counts = df['province'].value_counts()\n",
    "print('省份数量:', df['province'].nunique())\n",
    "province_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a30f567",
   "metadata": {},
   "source": [
    "### 小练习 4\n",
    "1. 统计城市数量 (nunique)。\n",
    "2. 输出所有城市名称列表 (unique)。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d431b4b8",
   "metadata": {},
   "source": [
    "## 5. 相关系数 corr (列之间线性关系)\n",
    "方法：df[['列A','列B']].corr() 返回相关矩阵 (-1 ~ 1)。\n",
    "解释：接近 1 → 同向强相关；接近 -1 → 反向强相关；接近 0 → 线性相关弱。\n",
    "示例：PM25 与 PM10 / NO2 与 SO2。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b5dc761b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "index",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "float64",
         "type": "float"
        },
        {
         "name": "PM10",
         "rawType": "float64",
         "type": "float"
        },
        {
         "name": "NO2",
         "rawType": "float64",
         "type": "float"
        },
        {
         "name": "SO2",
         "rawType": "float64",
         "type": "float"
        }
       ],
       "ref": "4e2769f8-f0f7-45c5-8201-4cb87c92e110",
       "rows": [
        [
         "PM25",
         "1.0",
         "0.9935886606019285",
         "0.9204083593913701",
         "0.9282190080647906"
        ],
        [
         "PM10",
         "0.9935886606019285",
         "1.0",
         "0.9518575637806216",
         "0.9449095821680138"
        ],
        [
         "NO2",
         "0.9204083593913701",
         "0.9518575637806216",
         "1.0",
         "0.9592134871163112"
        ],
        [
         "SO2",
         "0.9282190080647906",
         "0.9449095821680138",
         "0.9592134871163112",
         "1.0"
        ]
       ],
       "shape": {
        "columns": 4,
        "rows": 4
       }
      },
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PM25</th>\n",
       "      <th>PM10</th>\n",
       "      <th>NO2</th>\n",
       "      <th>SO2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PM25</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.993589</td>\n",
       "      <td>0.920408</td>\n",
       "      <td>0.928219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PM10</th>\n",
       "      <td>0.993589</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.951858</td>\n",
       "      <td>0.944910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NO2</th>\n",
       "      <td>0.920408</td>\n",
       "      <td>0.951858</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.959213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SO2</th>\n",
       "      <td>0.928219</td>\n",
       "      <td>0.944910</td>\n",
       "      <td>0.959213</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          PM25      PM10       NO2       SO2\n",
       "PM25  1.000000  0.993589  0.920408  0.928219\n",
       "PM10  0.993589  1.000000  0.951858  0.944910\n",
       "NO2   0.920408  0.951858  1.000000  0.959213\n",
       "SO2   0.928219  0.944910  0.959213  1.000000"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_matrix = df[['PM25','PM10','NO2','SO2']].corr()\n",
    "corr_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a0b57de",
   "metadata": {},
   "source": [
    "解读提示：相关 ≠ 因果；可能受第三变量共同影响。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "238155ba",
   "metadata": {},
   "source": [
    "### 小练习 5\n",
    "1. 找出相关系数最高的一对 (除去对角线)。\n",
    "2. 思考：为什么这两项可能相关？写一句猜测。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a6a8d72",
   "metadata": {},
   "source": [
    "## 6. 组合条件过滤 (逻辑与或)\n",
    "语法： (条件1) & (条件2) ; (条件A) | (条件B)。注意括号。\n",
    "示例：筛选 “PM25 > 55 且 NO2 > 30” 的记录。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "13d11043",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "index",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "date",
         "rawType": "datetime64[ns]",
         "type": "datetime"
        },
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "NO2",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "4460086b-0c9c-4de3-b73e-aaa5f6a2a9fb",
       "rows": [
        [
         "5",
         "2025-09-01 00:00:00",
         "北京",
         "60",
         "35"
        ],
        [
         "6",
         "2025-09-02 00:00:00",
         "北京",
         "65",
         "37"
        ],
        [
         "7",
         "2025-09-03 00:00:00",
         "北京",
         "58",
         "34"
        ],
        [
         "8",
         "2025-09-04 00:00:00",
         "北京",
         "62",
         "36"
        ],
        [
         "9",
         "2025-09-05 00:00:00",
         "北京",
         "66",
         "38"
        ]
       ],
       "shape": {
        "columns": 4,
        "rows": 5
       }
      },
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>city</th>\n",
       "      <th>PM25</th>\n",
       "      <th>NO2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2025-09-01</td>\n",
       "      <td>北京</td>\n",
       "      <td>60</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2025-09-02</td>\n",
       "      <td>北京</td>\n",
       "      <td>65</td>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2025-09-03</td>\n",
       "      <td>北京</td>\n",
       "      <td>58</td>\n",
       "      <td>34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2025-09-04</td>\n",
       "      <td>北京</td>\n",
       "      <td>62</td>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2025-09-05</td>\n",
       "      <td>北京</td>\n",
       "      <td>66</td>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        date city  PM25  NO2\n",
       "5 2025-09-01   北京    60   35\n",
       "6 2025-09-02   北京    65   37\n",
       "7 2025-09-03   北京    58   34\n",
       "8 2025-09-04   北京    62   36\n",
       "9 2025-09-05   北京    66   38"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filtered = df[(df['PM25'] > 55) & (df['NO2'] > 30)][['date','city','PM25','NO2']]\n",
    "filtered.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "5b20e289",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "index",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "date",
         "rawType": "datetime64[ns]",
         "type": "datetime"
        },
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "PM25",
         "rawType": "int64",
         "type": "integer"
        },
        {
         "name": "SO2",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "eca5e0a8-9b65-41bd-abf9-44cb6e6a1981",
       "rows": [
        [
         "0",
         "2025-09-01 00:00:00",
         "广州",
         "42",
         "7"
        ],
        [
         "1",
         "2025-09-02 00:00:00",
         "广州",
         "41",
         "7"
        ],
        [
         "2",
         "2025-09-03 00:00:00",
         "广州",
         "39",
         "6"
        ],
        [
         "3",
         "2025-09-04 00:00:00",
         "广州",
         "45",
         "7"
        ],
        [
         "4",
         "2025-09-05 00:00:00",
         "广州",
         "44",
         "7"
        ]
       ],
       "shape": {
        "columns": 4,
        "rows": 5
       }
      },
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>city</th>\n",
       "      <th>PM25</th>\n",
       "      <th>SO2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2025-09-01</td>\n",
       "      <td>广州</td>\n",
       "      <td>42</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2025-09-02</td>\n",
       "      <td>广州</td>\n",
       "      <td>41</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2025-09-03</td>\n",
       "      <td>广州</td>\n",
       "      <td>39</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2025-09-04</td>\n",
       "      <td>广州</td>\n",
       "      <td>45</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2025-09-05</td>\n",
       "      <td>广州</td>\n",
       "      <td>44</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        date city  PM25  SO2\n",
       "0 2025-09-01   广州    42    7\n",
       "1 2025-09-02   广州    41    7\n",
       "2 2025-09-03   广州    39    6\n",
       "3 2025-09-04   广州    45    7\n",
       "4 2025-09-05   广州    44    7"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 或条件示例：PM25 < 40 或 SO2 < 8\n",
    "either = df[(df['PM25'] < 40) | (df['SO2'] < 8)][['date','city','PM25','SO2']]\n",
    "either.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d08b0bbe",
   "metadata": {},
   "source": [
    "### 小练习 6\n",
    "1. 筛选出 PM10 > 80 或 PM25 > 60 的记录。\n",
    "2. 统计满足条件的城市有几个 (nunique)。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ddc82bd",
   "metadata": {},
   "source": [
    "## 7. Mini Case：提出并回答 3 个问题\n",
    "示例问题灵感：\n",
    "1. 哪个城市 5 天内 PM25 波动最大？(max - min)\n",
    "2. 哪个城市平均 NO2 最高？\n",
    "3. 是否存在 PM25 高但 SO2 低的城市天数较多？\n",
    "\n",
    "下面给出骨架代码，可按自己问题调整。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5764033e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "date",
         "rawType": "datetime64[ns]",
         "type": "datetime"
        },
        {
         "name": "PM25",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "0c9bfcb6-f1a4-4875-ae77-689976d0cfe3",
       "rows": [
        [
         "2025-09-02 00:00:00",
         "25"
        ],
        [
         "2025-09-04 00:00:00",
         "25"
        ],
        [
         "2025-09-05 00:00:00",
         "25"
        ],
        [
         "2025-09-01 00:00:00",
         "22"
        ],
        [
         "2025-09-03 00:00:00",
         "21"
        ]
       ],
       "shape": {
        "columns": 1,
        "rows": 5
       }
      },
      "text/plain": [
       "date\n",
       "2025-09-02    25\n",
       "2025-09-04    25\n",
       "2025-09-05    25\n",
       "2025-09-01    22\n",
       "2025-09-03    21\n",
       "Name: PM25, dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 1. 计算各城市 PM25 波动 (极差)\n",
    "pm25_range = df.groupby('city')['PM25'].agg(lambda x: x.max() - x.min()).sort_values(ascending=False)\n",
    "pm25_range.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "da2d9109",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "NO2",
         "rawType": "float64",
         "type": "float"
        }
       ],
       "ref": "47e6ef50-5545-4a1f-bd04-f8ae56294337",
       "rows": [
        [
         "北京",
         "36.0"
        ],
        [
         "重庆",
         "31.0"
        ],
        [
         "成都",
         "30.6"
        ],
        [
         "上海",
         "27.8"
        ],
        [
         "杭州",
         "20.2"
        ]
       ],
       "shape": {
        "columns": 1,
        "rows": 5
       }
      },
      "text/plain": [
       "city\n",
       "北京    36.0\n",
       "重庆    31.0\n",
       "成都    30.6\n",
       "上海    27.8\n",
       "杭州    20.2\n",
       "Name: NO2, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 2. 平均 NO2 最高城市\n",
    "no2_mean_city = df.groupby('city')['NO2'].mean().sort_values(ascending=False)\n",
    "no2_mean_city.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "ab3c2736",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.microsoft.datawrangler.viewer.v0+json": {
       "columns": [
        {
         "name": "city",
         "rawType": "object",
         "type": "string"
        },
        {
         "name": "0",
         "rawType": "int64",
         "type": "integer"
        }
       ],
       "ref": "d5ca9457-f1ab-436f-812a-8454ad627fc7",
       "rows": [
        [
         "成都",
         "3"
        ],
        [
         "北京",
         "1"
        ]
       ],
       "shape": {
        "columns": 1,
        "rows": 2
       }
      },
      "text/plain": [
       "city\n",
       "成都    3\n",
       "北京    1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 3. 统计 PM25 > 55 且 SO2 < 12 的记录数按城市汇总\n",
    "cond_counts = df[(df['PM25'] > 55) & (df['SO2'] < 12)].groupby('city').size().sort_values(ascending=False)\n",
    "cond_counts.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "504eeb2b",
   "metadata": {},
   "source": [
    "### 小练习 7\n",
    "写出你自己的第 4 个问题并回答 (例如：某指标平均值排名、某条件出现频次)。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ce3627b",
   "metadata": {},
   "source": [
    "## 8. 发现撰写模板\n",
    "使用如下结构写 3~5 条发现：\n",
    "- 指标 + 范围：广州 5 天 PM2.5 均值 42.2，日波动 6。\n",
    "- 排名/对比：北京 PM2.5 均值高于其他 4 城市 (≈+15)。\n",
    "- 条件频次：满足 “PM25>55 且 SO2<12” 的记录中，成都占 40%。\n",
    "- 相关观察：PM25 与 PM10 相关系数 0.xx，可能说明…… (非因果)。\n",
    "\n",
    "避免：仅罗列数字，无解释意义。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e322b9a4",
   "metadata": {},
   "source": [
    "## 9. 小结\n",
    "\n",
    "今日回顾：\n",
    "1. describe 提供快速概览\n",
    "2. 排序 + 分组统计支持对比\n",
    "3. 相关系数初步发现成对关系\n",
    "4. 组合条件过滤定位特殊记录\n",
    "5. 将计算转化为“可读发现”"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d591ea45",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CV 相对波动:\n",
      " city\n",
      "广州    0.057\n",
      "北京    0.054\n",
      "杭州    0.041\n",
      "上海    0.035\n",
      "成都    0.032\n",
      "Name: PM25, dtype: float64\n",
      "最高5行:\n",
      "          date city  PM25\n",
      "9  2025-09-05   北京    66\n",
      "6  2025-09-02   北京    65\n",
      "8  2025-09-04   北京    62\n",
      "18 2025-09-04   成都    62\n",
      "5  2025-09-01   北京    60\n",
      "最低3行:\n",
      "          date city  PM25\n",
      "23 2025-09-04   杭州    37\n",
      "20 2025-09-01   杭州    38\n",
      "2  2025-09-03   广州    39\n",
      "PM25>55 且 SO2<12 条件记录数: 4\n",
      "       50%\n",
      "city      \n",
      "上海    45.0\n",
      "北京    62.0\n",
      "广州    42.0\n",
      "成都    59.0\n",
      "杭州    39.0\n"
     ]
    }
   ],
   "source": [
    "# 探索进阶代码集合\n",
    "import pandas as pd\n",
    "df = pd.read_csv('../data/air_quality_timeseries.csv', parse_dates=['date'])\n",
    "\n",
    "# 1. 各城市 PM25 CV (相对波动)\n",
    "cv = df.groupby('city')['PM25'].agg(lambda s: round(s.std()/s.mean(), 3))\n",
    "print('CV 相对波动:\\n', cv.sort_values(ascending=False).head())\n",
    "\n",
    "# 2. nlargest / nsmallest\n",
    "top_pm25 = df.nlargest(5, 'PM25')[['date','city','PM25']]\n",
    "low_pm25 = df.nsmallest(3, 'PM25')[['date','city','PM25']]\n",
    "print('最高5行:\\n', top_pm25)\n",
    "print('最低3行:\\n', low_pm25)\n",
    "\n",
    "# 3. 条件计数示例\n",
    "cond_count = ((df['PM25'] > 55) & (df['SO2'] < 12)).sum()\n",
    "print('PM25>55 且 SO2<12 条件记录数:', cond_count)\n",
    "\n",
    "# 4. 分组 describe 抽取中位数列\n",
    "g_desc = df.groupby('city')['PM25'].describe()\n",
    "print(g_desc[['50%']].head())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f70d6746",
   "metadata": {},
   "source": [
    "### 探索进阶技巧 (统计 + 思路)\n",
    "1. 描述统计差异：对比分组 describe：df.groupby('city')['PM25'].describe().\n",
    "2. 变异系数 (CV)=std/mean：衡量相对波动；(示例) df.groupby('city')['PM25'].agg(lambda s: s.std()/s.mean()).\n",
    "3. 排名函数：nlargest / nsmallest 比排序+head 更高效。\n",
    "4. 透视表思路（未正式讲）：groupby 多维度时可以多列聚合成宽表。\n",
    "5. 条件计数：使用 (cond).sum()；跨两条件 = ((A)&(B)).sum()。\n",
    "6. 相关矩阵可视策略：后续可用热力图（Day5 也可尝试）。\n",
    "7. 复合指标示例：标准化 (值-均值)/std 后相加（了解即可）。\n",
    "8. 识别“异常记录”思路：极值、分位数、组合条件频次稀少。\n",
    "9. 生成“发现候选”清单：先批量打印排序/聚合结果，再人工筛选。\n",
    "10. 记录假设与验证：写 markdown：假设 → 使用的统计/过滤 → 支撑或反例。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f40abf7",
   "metadata": {},
   "source": [
    "## 10. 课后作业提示\n",
    "在 *homework_day4.ipynb*：\n",
    "1. 任选 3 个城市，计算它们 PM25 均值、标准差、极差并做表格。\n",
    "2. 找出 NO2 平均值最高与最低的城市并做一句解释。\n",
    "3. 自拟 2 个逻辑组合筛选条件并统计命中行数。\n",
    "4. 写 120 字发现总结（至少 3 条发现 + 1 条数据局限）。\n",
    "(可选) 5. 把你今天写的 1 个问题封装成函数，例如传入 city 返回 PM25 极差。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4991181c",
   "metadata": {},
   "source": [
    "---\n",
    "📌 提示：明天 (Day 5) 将进入可视化，请保留好本 Notebook 结果，方便直接画图。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
